Lossy data compression has numerous applications, especially in communications, broadcasting, entertainment, and security. Video compression is a challenging task, because large compression ratios are required to transmit high-quality and high-resolution pictures over existing communication channels. This task is even more challenging in the context of wireless and mobile communications, or real-time encoding of media.
The recently adopted International Telecommunication Union Telecommunication Standardization Sector (ITU-T) H.265/High Efficiency Video Coding (HEVC) standard (International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC), 23008-2:2013, “Information technology—High efficiency coding and media delivery in heterogeneous environments—Part 2: High efficiency video coding”, November 2013) declares a set of state-of-the-art video coding tools that provide a reasonable tradeoff between coding efficiency and computational complexity. An overview on the ITU-T H.265/HEVC standard is given in the article by Gary J. Sullivan, “Overview of the High Efficiency Video Coding (HEVC) Standard”, in The Institute of Electrical and Electronics Engineers (IEEE) Transactions on Circuits and Systems for Video Technology, Vol. 22, No. 12, December 2012, the entire content of which is incorporated herein by reference.
Similar to the ITU-T H.264/advanced video coding (AVC) video coding standard, the HEVC/H.265 video coding standard provides for a division of the source picture into blocks, e.g. coding units (CUs). Each of the CUs could be further split into either smaller CUs or prediction units (PUs). A PU could be intra- or inter-predicted according to the type of processing applied for the pixels of PU. In case of inter-prediction, a PU represents an area of pixels that is processed by motion compensation using a motion vector specified for a PU. For intra prediction PU specifies prediction mode for a set of transform units (TUs). A TU can have different sizes (e.g., 4×4, 8×8, 16×16 and 32×32 pixels) and could be processed in different ways. For a TU transform coding is being performed, i.e. the prediction error is being transformed with a discrete cosine transform (DCT) and quantized. Resulting quantized transform coefficients are grouped into coefficient groups (CGs), each CG having 16 quantized transform coefficients.
As noted above, the core tools of these standards or similar proprietary codecs to encode the blocks of picture are inter- and intra-prediction, spectrum-transformation (e.g., DCT or its integer approximation) and quantization. Inter- and intra-prediction tools are used to generate a prediction signal for a given block. At the encoder side, the difference between a source block and its prediction, the so-called residual signal, is transformed into their spectrum, i.e. source block pixels are represented by transform coefficients in frequency domain. Further, the coefficients are quantized. Non-zero and zero quantized transform coefficients are often referred to as significant and insignificant coefficients, respectively. All syntax elements including quantized transform coefficients and side information (e.g., intra prediction modes for intra-coding and motion vectors for inter-coding) are binarized and entropy encoded. The part of entropy encoded coefficients in a compressed H.265/HEVC bit-stream may exceed 80%.
The stages of encoding quantized transform coefficients are as follows.
Encoding the position of last significant coefficient, i.e. the last non-zero quantized transform coefficient.
Encoding the significance map that is used to restore positions of all the non-zero coefficients.
Sign encoding of the significant coefficients.
Magnitude encoding of the significant coefficients.
These stages are performed in the context of quantized transform coefficients being split into so-called CGs. Each CG is a subset that typically consists of 4×4 coefficients.
As noted above, state-of -the-art video coding standards are based on partitioning of a source picture (image) into blocks. Processing of these blocks depends on their size, spatial position and a coding mode specified by an encoder. Coding modes could be classified into two groups according to the type of prediction: intra- and inter-prediction modes. Intra-prediction uses pixels of the same image to generate reference samples based on which the prediction values for the pixels of the block being reconstructed are predicted. Intra-prediction could be also referred to as spatial prediction. Inter-prediction modes are designed for temporal prediction and uses reference samples of previous and/or next pictures to predict pixels of the block of the current picture.
Due to different type of redundancy, prediction processes for intra- and inter-coding are different. Intra-prediction typically constructs a one-dimensional buffer of reference samples. Inter-prediction uses sub-pixel interpolation of two-dimensional reference pixel matrix.
However, during standardization of ITU-T H.264/AVC a third technique for motion prediction has also been considered: motion compensated temporal filtering (MCTF). MCTF processes a three-dimensional (3D) array of pixels that is constructed by stacking up blocks of pixels. Each of these blocks is extracted from its own picture and stacking of these blocks is performed in the same order as specified for the pictures of the video sequence. The output of MCTF is two sets of blocks: high-pass blocks (residuals) and low-pass blocks. These blocks are further processed to produce prediction and residual blocks.
An overview on MCTF is provided in J. R. Ohm, “Complexity and delay analysis of MCTF interframe wavelet structures,” ISO/IEC JTC1/SC29/WG11, Document M8520, July, 2002. MCTF is implemented by employing a lifting process as exemplified in FIG. 18, in which the motion compensation features of H.26x video coding standards could be reused in MCTF prediction and update steps.
As shown in FIG. 18, for two consecutive pictures A and B of a video sequence a pyramid-based motion-compensated transform is performed to obtain “temporal” low pass frame L and high pass frame H. This “temporal” prediction could be written as:
            H      ⁢              (                  m          ,          n                )              =                  B        ⁡                  (                      m            ,            n                    )                    -                        (                      1            -            α                    )                ⁢                  (                      1            -            β                    )                ⁢                  A          ⁡                      (                                          m                -                k                            ,                              n                -                l                                      )                              -                        (                      1            -            α                    )                ⁢        β        ⁢                                  ⁢                  A          ⁡                      (                                          m                -                k                            ,                              n                -                l                +                1                                      )                              -                        α          ⁡                      (                          1              -              β                        )                          ⁢                  A          ⁡                      (                                          m                -                k                +                1                            ,                              n                -                l                                      )                              -              α        ⁢                                  ⁢        β        ⁢                                  ⁢                  A          ⁡                      (                                          m                -                k                +                1                            ,                              n                -                l                +                1                                      )                                                  L        ⁢                  (                      m            ,            n                    )                    =                                    A            ⁡                          (                              m                ,                n                            )                                +                                                    α                ⁢                                                                  ⁢                β                            2                        ⁢                          H              ⁡                              (                                                      m                    +                    k                    -                    1                                    ,                                      n                    +                    l                    -                    1                                                  )                                              +                                                    α                ⁡                                  (                                      1                    -                    β                                    )                                            2                        ⁢                          H              ⁡                              (                                                      m                    +                    k                    -                    1                                    ,                                      n                    +                    l                                                  )                                              +                                                                      (                                      1                    -                    α                                    )                                ⁢                β                            2                        ⁢                          H              ⁡                              (                                                      m                    +                    k                                    ,                                      n                    +                    l                    -                    1                                                  )                                              +                                                                      (                                      1                    -                    α                                    )                                ⁢                                  (                                      1                    -                    β                                    )                                            2                        ⁢                          H              ⁡                              (                                                      m                    +                    k                                    ,                                      n                    +                    l                                                  )                                                    =                              1            2                    ⁡                      [                                          α                ⁢                                                                  ⁢                β                ⁢                                                                  ⁢                                  B                  ⁡                                      (                                                                  m                        +                        k                        -                        1                                            ,                                              n                        +                        l                        -                        1                                                              )                                                              +                              α                ⁢                                  (                                      1                    -                    β                                    )                                ⁢                                  B                  ⁡                                      (                                                                  m                        +                        k                        -                        1                                            ,                                              n                        +                        l                                                              )                                                              +                                                (                                      1                    -                    α                                    )                                ⁢                                                                  ⁢                β                ⁢                                                                  ⁢                                  B                  ⁡                                      (                                                                  m                        +                        k                                            ,                                              n                        +                        l                        -                        1                                                              )                                                              +                                                (                                      1                    -                    α                                    )                                ⁢                                  (                                      1                    -                    β                                    )                                ⁢                                  B                  ⁡                                      (                                                                  m                        +                        k                                            ,                                              n                        +                        l                                                              )                                                              -                                                (                                      1                    -                    α                                    )                                ⁢                                  (                                      1                    -                    β                                    )                                ⁢                α                ⁢                                                                  ⁢                β                ⁢                                                                  ⁢                                  A                  ⁡                                      (                                                                  m                        -                        1                                            ,                                              n                        -                        1                                                              )                                                              -                                                (                                      β                    -                                          β                      2                                                        )                                ⁢                                  (                                      1                    -                                          2                      ⁢                                                                                          ⁢                      α                                        +                                          2                      ⁢                                                                                          ⁢                                              α                        2                                                                              )                                ⁢                                  A                  ⁡                                      (                                          m                      ,                                              n                        -                        1                                                              )                                                              -                                                (                                      1                    -                    α                                    )                                ⁢                                  (                                      1                    -                    β                                    )                                ⁢                α                ⁢                                                                  ⁢                β                ⁢                                                                  ⁢                                  A                  ⁡                                      (                                                                  m                        +                        1                                            ,                                              n                        -                        1                                                              )                                                              -                                                (                                      α                    -                                          α                      2                                                        )                                ⁢                                  (                                      1                    -                                          2                      ⁢                                                                                          ⁢                      β                                        +                                          2                      ⁢                                                                                          ⁢                                              β                        2                                                                              )                                ⁢                                  A                  ⁡                                      (                                                                  m                        -                        1                                            ,                      n                                        )                                                              +                                                (                                      2                    -                                                                  (                                                  1                          -                                                      2                            ⁢                                                                                                                  ⁢                            α                                                    +                                                      2                            ⁢                                                                                                                  ⁢                                                          α                              2                                                                                                      )                                            ⁢                                            ⁢                                              (                                                  1                          -                                                      2                            ⁢                                                                                                                  ⁢                            β                                                    +                                                      2                            ⁢                                                                                                                  ⁢                                                          β                              2                                                                                                      )                                                                              )                                ⁢                                  A                  ⁡                                      (                                          m                      ,                      n                                        )                                                              -                                                (                                      α                    -                                          α                      2                                                        )                                ⁢                                  (                                      1                    -                                          2                      ⁢                                                                                          ⁢                      β                                        +                                          2                      ⁢                                                                                          ⁢                                              β                        2                                                                              )                                ⁢                A                ⁢                                  (                                                            m                      +                      1                                        ,                    n                                    )                                            -                                                (                                      1                    -                    α                                    )                                ⁢                                  (                                      1                    -                    β                                    )                                ⁢                α                ⁢                                                                  ⁢                β                ⁢                                                                  ⁢                                  A                  ⁡                                      (                                                                  m                        -                        1                                            ,                                              n                        +                        1                                                              )                                                              -                                                (                                      β                    -                                          β                      2                                                        )                                ⁢                                  (                                      1                    -                                          2                      ⁢                                                                                          ⁢                      α                                        +                                          2                      ⁢                                                                                          ⁢                                              α                        2                                                                              )                                ⁢                                  A                  ⁡                                      (                                          m                      ,                                              n                        +                        1                                                              )                                                              -                            ⁢                                                (                                      1                    -                    α                                    )                                ⁢                                  (                                      1                    -                    β                                    )                                ⁢                α                ⁢                                                                  ⁢                β                ⁢                                                                  ⁢                                  A                  ⁡                                      (                                                                  m                        +                        1                                            ,                                              n                        +                        1                                                              )                                                                        ]                                ,  where α and β are weighting coefficients used for subpixel interpolation, and m, k, n, l are pixel coordinates within a respective picture.
It is known that picture coding introduces visual artifacts into a reconstructed image. These artifacts could be caused by quantization noise and blocking effects. Prediction techniques, such as intra- and inter-prediction, use already decoded/reconstructed samples as reference samples to predict a block of pixels, and therefore, noise inside this decoded information may influence the prediction accuracy. This noise may also imply higher energy of the residual signal for a block to be encoded, which may be addressed by increased bit-rate, or by accepting noticeable artifacts if increasing the bit-rate of the encoded stream is not acceptable.